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WHAT IS THE PROBLEM? 


As the statistics in the latest Annual Monitoring Report show, oversubscription has become a serious problem in 
Horizon 2020. We see a decrease in the success rate of eligible full proposals from an average of 21% in FP7 to 
13.2% in 2014 to finally 11.8% in 2015. The situation is even more dire for LEIT ICT (from 9.2% in 2014 to 6.7% in 
2015). To improve the likelihood of being selected, researchers spend increasing amounts of time on writing and 
optimizing proposals. However, their efforts are more and more likely to be wasted as we increasingly see cases 
of proposals being rejected with an evaluation score at the maximum available score (e.g. 14.5/15). 
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The graph below shows that over FP7 and Horizon 2020 private sector participation (including SMEs) has 
remained between 25 and 30%! in terms of funding, after a steep decline since FP4. However, we fear that 
oversubscription coupled with a low perception of the evaluation process will deter industry from submitting 
proposals. The result will be a decrease in industry participation, thus depriving other participants from the 
benefits that industry brings to collaborative research and innovation and preventing the European Commission 
from fulfilling its objectives. 
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As the success rates decrease, it is even more important to ensure a consistent and high quality of proposal 
evaluation and applicants’ trust both in the outstanding expertise of the evaluators and in the evaluation results. 
In 2015, only 16% of evaluators were selected from the private sector, including SMEs (a breakdown of the 
statistics per call/strategic objective and precise organisational type might be useful). This is an alarming decrease 
from the previous year when 21. 9% of evaluators were drawn from the private sector, especially considering 


that this number comprises a growing number of consultants?. 


In our experience, it is not necessarily a lack of Industry experts being offered that is an issue but the selection of 
experts. As the programme puts increased emphasis on innovation, it is crucial that evaluators contribute the 
necessary knowledge to assess the impact of projects. They should ensure that the selected projects serve the 
interest of the markets in the most efficient way and that projects have the necessary elements to have impact 
in a very competitive environment. Starting with the instruction of reviewers on the application of the evaluation 
criteria, the evaluation process should lead to the selection of projects with real impact, particularly in Industry- 
driven PPPs. 


WHAT ARE DIGITALEUROPE’s RECOMMENDATIONS? 


1. Focus in the Work Programmes 


Strategic objectives in the forthcoming Work Programme should be more focused. The scope of calls must be 
narrowed, meaning that the challenge needs to be more focused and specific on topics to be addressed and on 
relevant expected impacts. Where a call includes multiple topics and impacts, it should be made explicit to which 
extent they are essential, complementary and/or alternative. If partial coverage is to be reflected in lower scoring, 
relevant criteria should also be made explicit to evaluators and proposers. This will help to channel efforts towards 
the most strategic research and innovation areas. Ideally, all impacts should be measurable and relevant metrics 
should be provided in the call. Considering LEIT PPPs, it is important to maintain the role of a broad-based industry 
representation from definition to implementation of the PPPs. 





2 French position on the Horizon 2020 Interim Evaluation 
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2. Two stage procedures 


In some cases, and in addition to the above, two-stage submissions might provide a mechanism to help address 
the oversubscription problem, though at the risk of prolonging the time-to-grant, which does not fit the very 
dynamic nature of the ICT sector, and of further increasing oversubscription in the first stage. If used, the initial 
length of a proposal to be submitted in stage 1 should not exceed 20 pages. In addition, a strong filtering process 
should take place, so that the first stage becomes more selective (leading to a success rate of at least 33% or even 
50% in the second stage). Because of this, however, quality, fairness and transparency of the first stage evaluation 
process needs to be at the same high level of the second stage. 


3. Industry engagement in the evaluation process 


Industry commits to continue making available highly qualified people as experts for the evaluations and ensuring 
that being an evaluator in an EU Research & Innovation Programme is an asset to one’s professional reputation 
and career experience. The representation of industry experts among evaluators needs to be in line with industry 
participation in projects. The European Commission should thus aim at a private sector participation rate of at 
least 30% for the first Call of the next Framework Programme after a gradual increase in the remaining years of 
H2020 from the current 16%. Considering the on-going processes of digitisation in all areas of economy and 
society, we recognise the important role of the digital industry. DIGITALEUROPE therefore gladly takes on the 
responsibility to raise awareness of this need among both our own members and other industry sectors to 
promote the participation of industry experts in evaluations. 


4. Selection of experts 


In return, the European Commission needs to strengthen its efforts to encourage and enable the participation of 
Industry in the evaluation of projects. We see a need for simplifying the criteria for identification of experts, 


recognising the value of the industry expertise and reconsidering undue fears about conflicts of interest>, 
especially for the evaluation of close-to-market proposals. In our view, conflicts of interest are certainly not only 
an industry-specific issue. The European Commission should interpret the rules transparently and equally to 
industry and academia experts. In turn we emphasise that experts, when called, will be acting on a strictly 
personal basis and not on behalf of their companies. 


5. Diversify evaluators’ profiles in the panel 


The European Commission should also pay attention to evaluator’s profiles. Taking account of the types of 
projects expected in the calls, profiles to assess technology research, as well as profiles to assess usage and take- 
up innovation should be selected to ensure balanced assessments. To ensure the necessary representation of 
expertise, an enlargement of the panel size could be considered as well. 
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6. Use of remote consensus 


One of the oft-mentioned obstacles for the participation of industry experts in evaluations is the lack of time for 
physical travel to Brussels. We already see the use of remote evaluation, which should be implemented widely 
across the Programme. In addition, we very much support the increased choice of remote consensus meetings. 
This would also provide an efficient mean of implementing the two above recommendations, since it would allow 
to employ in these two phases a larger number of experts, not all of which will be required to attend the on-site 
panel meetings for final assessment and ranking of the proposals. In addition, more time could be dedicated on- 
site for discussing minority reports and for harmonizing evaluations. 


7. Selection of Recorders 


It is crucial that recorders can follow the discussions and clearly capture the decisions taken, avoiding any 
ambiguity. On occasion, we have also noticed a lack of English language skills. Thus, recorders with proven writing 
skills, in addition to a capacity to understand the issues at stake, should be preferred. These skills are even more 
critical in remote consensus meetings. 


8. Evaluation feedback 


Besides the composition of the evaluation teams, we also recommend to consider the quality of the evaluation 
and especially the feedback received by beneficiaries. As even proposals with marks of 14.5/15 are not selected, 
it is more important than ever that the criteria/reasons for the ranking are communicated clearly to the 
applicants. Although specific comments/examples are useful, their relevance to the final scoring is not always 
evident. Without re-introducing formal sub-scoring, it is recommended to start each sub-criterion with a short 


yout 


sentence that clearly states whether the proposal addresses it “poorly”, “well”, “very well” or “extremely well”. 


For more information please contact: 
Annika Eberstein, DIGITALEUROPE’s Policy Manager 
+32 2 609 53 36 or annika.eberstein @digitaleurope.org 
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ABOUT DIGITALEUROPE 


DIGITALEUROPE represents the digital technology industry in Europe. Our members include some of the world's largest IT, 
telecoms and consumer electronics companies and national associations from every part of Europe. DIGITALEUROPE wants 
European businesses and citizens to benefit fully from digital technologies and for Europe to grow, attract and sustain the 
world's best digital technology companies. 


DIGITALEUROPE ensures industry participation in the development and implementation of EU policies. DIGITALEUROPE’s 
members include 61 corporate members and 37 national trade associations from across Europe. Our website provides 
further information on our recent news and activities: http://www.digitaleurope.org 
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Corporate Members 





Airbus, Amazon Web Services, AMD, Apple, BlackBerry, Bose, Brother, CA Technologies, Canon, Cisco, Dell, Dropbox, Epson, 
Ericsson, Fujitsu, Google, Hewlett Packard Enterprise, Hitachi, HP Inc., Huawei, IBM, Intel, iQor, JVC Kenwood Group, Konica 
Minolta, Kyocera, Lenovo, Lexmark, LG Electronics, Loewe, Microsoft, Mitsubishi Electric Europe, Motorola Solutions, NEC, 
Nokia, Nvidia Ltd., Océ, Oki, Oracle, Panasonic Europe, Philips, Pioneer, Qualcomm, Ricoh Europe PLC, Samsung, SAP, SAS, 
Schneider Electric, Sharp Electronics, Siemens, Sony, Swatch Group, Technicolor, Texas Instruments, Toshiba, TP Vision, 
VMware, Western Digital, Xerox, Zebra Technologies. 


National Trade Associations 


Austria: |OO Germany: BITKOM, ZVEI Slovakia: ITAS 

Belarus: INFOPARK Greece: SEPE Slovenia: GZS 

Belgium: AGORIA Hungary: IVSZ Spain: AMETIC 

Bulgaria: BAIT Ireland: ICT IRELAND Sweden: Foreningen 
Cyprus: CITEA Italy: ANITEC Teknikforetagen i Sverige, 


Denmark: DI Digital, IT-BRANCHEN 
Estonia: ITL 

Finland: TIF 

France: AFNUM, Force Numérique, 
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Switzerland: SWICO 

Turkey: Digital Turkey Platform, ECID 
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